cloudstate
cloudstate copied to clipboard
cloudstateio
Forward and Effect to Remote HTTP Services
Hello guys!
Would it be interesting to add a forward and effects mechanism for remote http services? In this way, a cloudstate service could forward a message to external APIs. Let me know what you think.
Cheers
@sleipnir Do you mean instead of having the serverless function do an async HTTP-request itself?
Yes @viktorklang. Imagine that I have to send a request there is something external to my cloudstate environment, like an external API of some client of mine sending a hook for example, in this case I need to make an http request but I don't want to have to worry about this underlying infrastructure.
@sleipnir Hmm, it would be rather straightforward, but on the other hand you sort of enter the durable distributed statemachine place (especially if you're trying to build a DAG of external interactions (think Saga Pattern).
Hi @viktorklang !
@sleipnir Hmm, it would be rather straightforward, but on the other hand you sort of enter the durable distributed statemachine place (especially if you're trying to build a DAG of external interactions (think Saga Pattern).
Yes, relatively simple. And yes, I agree about SAGA because it would basically be the SAGA pattern based on orchestration, with the proxy being the orchestrator. We could obviously start with the simplest cases and see what others think.
But I would love it because I could switch my production systems to Cloudstate faster :) Lol
@sleipnir You could probably implement something in the API right now and iterate on the end user experience while having the requests performed in the user function rather than in the proxy, then move the functionality seamlessly into the proxy?
@viktorklang sorry but i don't think i understand the question. Do you want to know about my specific use case, whether it would be possible for me to make the necessary calls within the user functions? Or are you asking me about transparency?
@sleipnir I think there are two components to this: 1) API for end users, in the different User Language Support 2) Proxy implementation & protocol definition. 1 can be implemented separately by stubbing out the need for proxy support by instead invoking something like an HTTP API under the hood. It would be interesting to contrast and explore the value of implementing it inside the proxy vs just having calls made from within the User Language Support.
Does that make more sense?
@viktorklang I believe this is a fully proxy feature. Delegating this to the user's language consistently across all languages would be very difficult. I think that all the mechanisms and tools to do this are already in the proxy and in the protocol, I believe it is more a matter of putting the pieces together. But both approaches have potential. What would you like to try first?
Maybe it would be interesting to mention this in the call of the collaborators and to hear a little from the others about what they think
@sleipnir I agree on the proxy feature, but what I'm saying is that it would be interesting to explore the user API first. Then once that has been explored it is much easier to implement it on the proxy side
Ok @viktorklang after your last comment I understood that. I just made one more comment about what I think. As I mentioned, I have no problem trying any approach.
Hello @viktorklang what do you think of the API proposal (in this case in Java) below:
// #lookup
private final ServiceCallRef<Protocol.HttpRequest> requestRef;
private final ServiceCallRef<Protocol.HttpResponse> responseRef; //Optionally we can define a commandHandler to handle the response
public ShoppingCartEntity(Context ctx) {
responseRef =
ctx.serviceCallFactory()
.lookup("example.shoppingcart.ShoppingCartService", "ItemAddedToCart", Hotitems.ItemAdded.class); //Note that here we pass the type of the class that will be sent in the body of HttpResponse.
requestRef =
ctx.serviceCallFactory()
.lookup(
"https://some.url:4443/some/resource", responseRef);
// #fire-and-forget
requestRef =
ctx.serviceCallFactory()
.lookup(
"https://some.url:4443/some/resource"); // Without any response reference this call will be asynchronous
// and without any commandHandler to be called later for this request
// #fire-and-forget
}
// #lookup
class CommandHandlerWithForward {
// create request headers if use
Map<String, Object> headers = new HashMap();
headers.put("Custom-Header", "Custom-Value");
// #forward
@CommandHandler
public void addItem(Shoppingcart.AddLineItem item, CommandContext ctx) {
// ... Validate and emit event
ctx.forward(
requestRef.createCall(
Protocol.HttpRequest.newBuilder()
.setHeaders(headers)
.setBody(
// Obviously we have to note the protobuf in some way so that this object can be serialized via json.
Hotitems.Item.newBuilder()
.setProductId(item.getProductId())
.setName(item.getName())
.setQuantity(item.getQuantity())
.build())
));
}
// #forward
}
In this case, I kept the user interface as close as possible to the existing forward and effects interface. I just imagined some wrapper classes to handle the request and response and some changes to the lookup interface. This would also eliminate the need to implement a more complex state machine (like the one needed for an implementation of the SAGA pattern), not that this is not interesting in the future, but we could start with the simplest and let the user if he wishes to implement the standard SAGA by itself, since we would have exposed an api that would allow it to create more complex flows. Just building blocks for now.
Yeah I've thought about this. We definitely want/need to support something like this. What I do want to make sure is that the abstractions we provide are the right ones.
Of course, there's nothing stopping you from calling a remote service directly. It does mean that the proxy is not able to orchestrate it, but at least there's nothing blocking implementing something like this today.
One thing that I'm not sure about is how useful the existing forwarding support is, I'd like to validate that both with some example use cases, as well as seeing people use it to solve real problems. One problem with the support today is, what if what you actually want is not to forward, but rather, to delegate, and then handle the response? One way to do that would be to forward to the remote service, then have it forward back. But I'm not sure that that design makes sense, it means there's a two way awareness of each other. So I think a new abstraction that was maybe "forward to this service (local or remote), and then handle the response from that service with this other call". But that's also getting very complex, and how do you maintain context between those? While it does mean the proxy is orchestrating everything, for the developer, it may feel very unnatural to do that, and the advantages of the proxy orchestrating might not outweigh that complexity.
Yeah, agree that thinking through realistic use cases would help a lot here.
Also agree about questioning whether the extra complexity outweighs what is gained. Any limitations in doing this through the protocol could be frustrating, rather than just orchestrating with async calls directly. But if the proxy added extra capabilities automatically (like circuit breakers for remote services) then it could make more sense. I expect the response pattern to tie together contexts would become similar to actor-like ask/pipe pattern — make this request and then call me on this handler with the response plus this context.
There's a temptation to simply offer actor support here, and maybe that's a good idea? But it also might not be a good idea - Cloudstate is trying to offer higher level functionality than actors, I believe. But perhaps built in to the protocols we could have the basics of actor support (tell, forward, ask/pipe), and then use the user support libraries to expose those as higher level features, eg, a user support library could provide an HTTP client abstraction and build on top of those basic actor constructs.
I think that in general there is a tendency towards complexity. But I wonder if what basically the proxy should offer from the user's perspective is the ability to simply route calls and that the user is able to forward successful responses or not to the respective user-defined handlers. At most, the proxy could expose backoff and retry strategies similar to the RetrySink policy on top of this routing capability. The rest would be left to the end users. Certainly a world of possibilities would open up if only this routing capacity was added. In short, let's start simple and add complexities on demand
Agree that it would be good to support, and I also see it as tending towards supporting underlying actor capabilities in general. It may just be in the language support modules, not in the proxy and protocols, but it looks like that's where it would be headed.
Here's why I see that: the function could make a sync request to a service, but we probably wouldn't want to encourage blocking in the function. The function could make an async request to a service and process the response in a callback, but users could also expect to access the original command context or modify local state in the callback — creating the same problems with doing this with futures in an actor. The function needs to encapsulate state in the same way as actors, so we then end up wanting to support and encourage an ask/pipe (with underlying tell), either implemented just locally in the language support or via the protocol. Mixing in other concurrency mechanisms in the function could be problematic, so we may want to cover use cases like this directly.
Depends on how we define simple as to what starting simple means. Starting with the minimum required for a simple use case, eg, to allow proxying HTTP requests, might be simple for us to implement now, but the challenge is that what we produce will very likely not be compatible with the next level of feature that we want to offer, so to offer that we'll have to add a new mechanism to the protocol, and over time these will add up, and we'll end up with 10, 20, 30 purpose built mechanisms for communication, leading to a very complex protocol, making it difficult to implement all the support libraries, etc - not simple at all.
In contrast, starting by offering a simple but complete mechanism, ie, a protocol that exposes basic actor constructs, may be more complex now because it requires more thinking and design, and may require more work in the proxy and support libraries initially, and might be more complex to use initially, but in the long term will be far simpler, if we get it right, no changes fundamental to the protocol will be needed to add new features, it will be straight forward for support library implementers to implement and maintain, and so on.
@jroper I meant simple from the perspective of the end user. I say simple abstractions for the user. And of course I spoke of a simple, general-purpose use case as a starting point. I agree with your point of view to establish a good design from the beginning will facilitate future additions. But it is a difficult measure to get right, when the design is good enough to start some implementation, my only fear is to give up a feature because we think it would be complex without even trying to establish a use case. But from what you said that would not be the case. @pvlugter I don't know if this should be in the support language. I think that at the end of the day an external request is still a "state" in transit (think about REST and its state on the client side) and the proxy should deal with any type of state in my opinion