S3AsyncClient integration

[rolkhas2]

Type: Feature I'm currently using S3AsyncClient with Project Reactor and Spring Boot. This project seems interesting, but it's kind of irrelevant to me if it's based on slow-synchronous network interactions. If there's an option of implementing classes to wrap fast S3 connections (e.g. async), I'd like to use it.

Is your feature request related to a problem? Please describe. From what I understand, there's only support for slow S3 connections, which is not ideal for me (I'm writing fast and IO-based software with Reactor).

Describe the solution you'd like S3AsyncClient support would be great.

Describe alternatives you've considered I don't know of any Publisher which AWS SDK 2 returns, so there's no alternatives I know of.

[maciejwalkowiak]

Just merged SQS implementation uses non-blocking S3AsyncClient under the hood. Perhaps @tomazfernandes wants to add some comments.

[tomazfernandes]

Well, I may be missing something, but the issue is regarding the S3AsyncClient rather SqsAsyncClient.

Regardless, I agree that this being a SDK 2.0 solution, it might be interesting offering async counterparts to blocking methods for the integrations.

As far as I've seen, the AsyncClients offer exactly the same API's as the blocking Clients, the only difference being the return method returns a CompletableFuture<Response> rather than Response.

So we might simply offer the AsyncClients instead of the blocking ones, offering the async methods in the templates and using the same logic but with calling .join() in the blocking ones.

Although that might not be ideal for users that want to use the blocking clients directly.

[maciejwalkowiak]

Oh boy I should have had coffee before going through github notifications 🤦‍♂️

It fits under general theme of providing async/reactive clients. Nothing to do with SQS ..

[maciejwalkowiak]

@tomazfernandes I guess there is a performance penalty for using async clients in sync manner (with join) vs pure sync clients in sync scenarios, right?

We can also autoconfigure both clients and users will just inject they want. I am only not sure if sync clients do not create connections on instantiation - if so this is not an option.

[tomazfernandes]

@tomazfernandes I guess there is a performance penalty for using async clients in sync manner (with join) vs pure sync clients in sync scenarios, right?

Hmm, not so sure about that. These are mainly IO operations, and I don't think NIO / event loop connections should be any slower than blocking ones. Also join is not a busy wait, so there should be really no CPU usage while waiting. So IMHO I wouldn't really use this as a criteria.

I am only not sure if sync clients do not create connections on instantiation - if so this is not an option.

Yeah, not sure about that either. But AFAIK, if e.g. an app doesn't have permissions to SQS, it won't throw an error on startup, but rather when an action is executed. So my guess would be that the client has connections with a max-alive setting rather than a pool of long lived connections.

We can also autoconfigure both clients and users will just inject they want.

Yeah, that's a tricky one. From what I've learned when creating auto configuration for the RetryableTopic feature, the goal of auto configuration is to provide a minimum set of opinionated infrastructure with sensible defaults, some of them being overridable through properties. Anything more complex should be handled by user code in @Configuration classes, with auto configuration backing off.

Also, I don't think it's common to provide bare beans such as these Clients. Usually, we'd have abstractions such as xxxClientFactory, and provide the xxxTemplate bean. This way the container can be in charge of the complete client lifecycle, and add stuff like event listeners, interceptors, etc.

Long story short, I think one option would be to provide async methods in the xxxTemplates, and use the Async client internally with the join methods in the blocking variants.

Another option would be autoconfiguring a specialized xxxAsyncTemplate and not necessarily expose an AsyncClient to users. Given bean instantiation is lazy, if users don't inject the xxxAsyncTemplate, it shouldn't even be instantiated, right?

For SQS, we depend on the AsyncClient for infrastructure, so it should be a matter of deciding whether to provide the blocking SqsClient. We can think about that when creating the SqsTemplate.