Questions on codegen and HTTP behavior for optional streaming shapes

[david-perez]

I gather from this test that optional streaming blob shapes are valid in Smithy. How are these to be interpreted regarding codegen and in HTTP-based protocols?

In my opinion, a strict interpretation of the spec would be:

• At codegen time, generate optional types for non-required blob streaming shapes (e.g. Option<ByteStream>).
• The HTTP RFC differentiates between HTTP messages with no body (no Content-Length or Transfer-Encoding header) and HTTP messages with an empty body (Content-Length: 0). Therefore, if no value is provided (None in Rust) for an operation output member, no HTTP body should be sent.

Is this interpretation correct?

---

In any case, I think allowing for this distinction between no HTTP body and empty HTTP body is not worth having.

Some data points:

• Currently, the Rust client SDK disregards optional blob streaming shapes, instead interpreting them as if they were always required.
• In the Java SDK, prior to SDK version 2.17.117, empty responses from lambda's invoke operation were exposed as an empty byte array. In 2.17.117, a behavior change was released to fix an unrelated bug. The behavior change had the unintentional effect of changing the empty byte array of the operation to be returned as null.

I think that allowing for the distinction will only be a source of bugs. I'd like the possibility of the spec being amended to always pair required with streaming (which is already coupled with httpPayload) be considered.

[david-perez]

Actually, the issue still applies with a non-streaming member e.g. optional httpPayload-bound members are allowed. What are their HTTP semantics?

---

In restJson1 it's impossible to make the distinction, because server implementations should always respond with a JSON object, even if the output parameters are empty. There's a test checking that that also puzzled me in #1068.

[david-perez]

The HTTP RFC differentiates between HTTP messages with no body (no Content-Length or Transfer-Encoding header) and HTTP messages with an empty body (Content-Length: 0). Therefore, if no value is provided (None in Rust) for an operation output member, no HTTP body should be sent.

This is only in the case of requests. Today I learnt that for responses, things get more complicated:

For response messages, whether or not a message-body is included with
a message is dependent on both the request method and the response
status code ([section 6.1.1](https://datatracker.ietf.org/doc/html/rfc2616/#section-6.1.1))

[david-perez]

This was addressed in the case of streaming blob shapes in IDL v2 (https://smithy.io/2.0/guides/model-translations/migrating-idl-1-to-2.html#add-the-default-trait-to-streaming-blobs) by forcing the user to add @required or @default, so I'm going to close this and open a new issue specifically for non-streaming shapes: https://github.com/smithy-lang/smithy/issues/2311.