localstack

localstack copied to clipboard

Is there an existing issue for this?

• [X] I have searched the existing issues

Enhancement description

Please add LAMBDA_FORWARD_URL functionality to the new lambda implementation. The full version of the story is discussed here. But the short version is, it's essential for any service that has any dependency on an AWS Lambda function to be able to mock AWS Lambda function during integration tests in the simplest way; i.e. forwarding requests to an HTTP mock server (which is pretty typical for any services with external dependencies).

🧑‍💻 Implementation

No response

Anything else?

Let me know if you more information.

Hello 👋! It looks like this issue hasn’t been active in longer than five months. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

I put a reaction on this last week and it still closed it?

hey @joshorr, thanks! reopening the issue

Marking this feature request as accepted such that it does not get closed by the stale bot anymore.

It's a reasonable feature request for the reasons discussed in this Discuss post. The current workaround is to use a dedicated Lambda function as a forwarder.

Hi @akefirad and @joshorr 👋

An alternative solution for forwarding certain requests to real AWS is our LocalStack Extension called AWS replicator. You can check out a demo of the AWS replicator extension in yesterday's Serverless Office Hour event here.

I think such a generic solution that works for everything in LocalStack would be much more favorable than a service-specific workaround such as LAMBDA_FORWARD_URL. The AWS replicator extension could be used as a blueprint for a mocking proxy.

What do you think?

@joe4dev thanks for the update. I'm not sure. Let me try;

1. The extension is not an ideal solution; for start (if I understand it correctly) it's a proxy for AWS real resources. Our use case is not that. We'd like to forward the lambda calls to a mock server (e.g. WireMock, that's what I'm doing currently). Moreover, the extension seems to be an overkill for such a simple and basic (yet critical) use case. And I'm not even talking about the cost issue 🙂
2. If I may, I'd like to take a step back and understand the reasoning behind removing the LAMBDA_FORWARD_URL (I tried to find some info before, but to no avail). My understanding is that the main purpose of LocalStack is to make AWS services (i.e. app external dependencies) locally available only for testing (dependencies which otherwise would have been mocked one way or another). In other words, LocalStack is mocking AWS services. As an application developer, I don't wanna know how AWS services are mocked, I just want them to behave like real AWS service (to a reasonable extend), with low cost (i.e. resources, time, etc) as much as possible. For AWS lambdas, the simplest solution that comes to my mind is a simple forward proxy. It can't get any simpler than that, yet it perfectly serves the purpose.
3. You might have some stats to argue against this, but TBH I don't understand the value of running actual lambda functions inside LocalStack. Why would I need such functionality? To test how my code runs on AWS Lambda platform? But it's not AWS Lambda platform, so what's the point? For integration test? Sure, but I'm guessing most lambda developers use some framework to develop their lambdas (e.g. SAM, Serverless, etc) all of which are capable of starting the lambda similar to the AWS lambda env. So that can definitely be used for integration tests along side LocalStack which mocks other AWS service.
4. I'm sure there's other use cases solved but the mentioned solutions, but IMHO none of them solves this specific problem the way a simple forward proxy does. Deploying a user-defined lambda that acts a proxy can do the job, but it makes tests slower and also last time I tried the networking configuration was difficult to correctly setup (for me).

Thanks.

Thank you for the quick and helpful response @akefirad 🙏

1. I agree that the AWS Replicator Extension does not solve your WireMock forwarding scenario because the proxy mechanism has AWS as the hardcoded target. Adding a proxy feature (with a similar resource filtering configuration such as the AWS Replicator Extension) would generically solve this scenario for every LocalStack service.
2. I tried to answer the question regarding the motivation why we did not prioritize re-implementing the forwarding feature in our Lambda provider rewrite in the Discuss post here:

   The feature was initially introduced to mitigate limitations in the old Lambda implementation. Our new Lambda implementation achieves (almost) perfect feature coverage and parity with AWS following meticulous parity testing. Hence, forwarding Lambda requests was not a priority in the re-implementation.

3. Have you tried our Lambda Tools for improved developer experience such as hot-reloading (tutorial video), which enables super fast feedback cycles? LocalStack also offers many more features for Security Testing, Chaos Engineering, Cloud Sandboxes, State Management that go beyond what's possible in the real cloud. Full local integration testing and development is limited with other frameworks because their scope and emulation fidelity do not match the service coverage and AWS parity at LocalStack.
4. I agree that the forwarder Lambda is not ideal yet and a generic forwarding proxy for LocalStack could solve this scenario. Regarding networking, I encourage you to give it another try with the major improvements we shipped with LocalStack v3+. You can check out our blog post How we are making connecting to LocalStack easier.