Add support for S3 Object Lambda WriteGetObjectResponse

[duarten]

Bug Report

Version

├── aws-endpoint v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   └── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│       ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
├── aws-sdk-s3 v0.0.12-alpha (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   ├── aws-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   ├── aws-endpoint v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   ├── aws-http v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   │   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   ├── aws-hyper v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   │   ├── aws-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   ├── aws-endpoint v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   ├── aws-http v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   ├── aws-sig-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   │   │   ├── aws-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   │   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   ├── protocol-test-helpers v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   │   ├── smithy-client v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   │   │   ├── protocol-test-helpers v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   │   ├── smithy-http-tower v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
│   │   │   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   │   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   ├── smithy-http-tower v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   │   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   ├── aws-sig-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)
│   └── smithy-xml v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957)
├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.12-alpha#d61aa957) (*)

Platform

Lambda function.

AWS Services

S3.

Description

The write_get_object_response API results in an error:

Err(ServiceError { err: WriteGetObjectResponseError { kind: Unhandled(Error { code: Some("MethodNotAllowed"), message: Some("The specified method is not allowed against this resource."), request_id: Some("M02G85BHEAWFJTBM"), extras: {"s3_extended_request_id": "Iq+h8RBuphv1LmUd7OE6lA9KF5MF9xDmx33OdU4Ca89KpD6Uz8Jlu1Lz/8sXPZiXWwtNhGoYS2M="} }), meta: Error { code: Some("MethodNotAllowed"), message: Some("The specified method is not allowed against this resource."), request_id: Some("M02G85BHEAWFJTBM"), extras: {"s3_extended_request_id": "Iq+h8RBuphv1LmUd7OE6lA9KF5MF9xDmx33OdU4Ca89KpD6Uz8Jlu1Lz/8sXPZiXWwtNhGoYS2M="} } }, raw: Response { status: 405, version: HTTP/1.1, headers: {"x-amz-request-id": "M02G85BHEAWFJTBM", "x-amz-id-2": "Iq+h8RBuphv1LmUd7OE6lA9KF5MF9xDmx33OdU4Ca89KpD6Uz8Jlu1Lz/8sXPZiXWwtNhGoYS2M=", "allow": "HEAD, DELETE, GET, PUT", "content-type": "application/xml", "transfer-encoding": "chunked", "date": "Tue, 27 Jul 2021 01:18:17 GMT", "server": "AmazonS3", "connection": "close"}, body: SdkBody { inner: Once(Some(b"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>MethodNotAllowed</Code><Message>The specified method is not allowed against this resource.</Message><Method>POST</Method><ResourceType>OBJECT</ResourceType><RequestId>M02G85BHEAWFJTBM</RequestId><HostId>Iq+h8RBuphv1LmUd7OE6lA9KF5MF9xDmx33OdU4Ca89KpD6Uz8Jlu1Lz/8sXPZiXWwtNhGoYS2M=</HostId></Error>")), retryable: true } } })

I think this is because the S3 endpoint is not being correctly set. It should be s3-object-lambda.{region}.amazonaws.com, but WriteGetObjectResponseInput::make_operation doesn't seem to account for this.

I tried to set it manually by implementing ResolveAwsEndpoint, but unfortunately it's impossible to build an AwsEndpoint outside of its crate safely.

[rcoh]

your diagnosis is correct. We haven't yet implemented the custom behavior required for S3 object lambda. To work around this issue, you can manually construct the S3 partition resolver in a way that substitutes s3-object-lambda:

    let resolver = aws_endpoint::PartitionResolver::new(
        aws_endpoint::Partition::builder()
            .id("aws") 
            // if you need other regions, you can find the patterns in https://github.com/awslabs/aws-sdk-rust/blob/main/sdk/s3/src/aws_endpoint.rs
            .region_regex(r#"^(us|eu|ap|sa|ca|me|af)\-\w+\-\d+$"#)
            .default_endpoint(aws_endpoint::partition::endpoint::Metadata {
                uri_template: "s3-object-lambda.{region}.amazonaws.com",
                protocol: aws_endpoint::partition::endpoint::Protocol::Https,
                signature_versions: aws_endpoint::partition::endpoint::SignatureVersion::V4,
                 // important that to also override the credential scope here
                credential_scope: aws_endpoint::CredentialScope::builder().service("s3-object-lambda").build(),
            })
            .regionalized(aws_endpoint::partition::Regionalized::Regionalized)
            .build().expect("valid partition"), vec![]);
        
    let config = Config::builder().endpoint_resolver(resolver).build();
    let s3_client = s3::Client::from_conf(config);

[rcoh]

I've co-opted this issue to track implementation of the required custom behavior, we'll keep this up to date as this work is prioritized and completed

[rcoh]

this is pending release (0.53 contains a partial fix but a bug was discovered)

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.