localstack-extensions icon indicating copy to clipboard operation
localstack-extensions copied to clipboard

remove use of http2_server in replicator aws proxy

Open thrau opened this issue 1 year ago • 4 comments
trafficstars

Motivation

:construction:

Fixes https://github.com/localstack/localstack-extensions/issues/45

This is an attempt to move away from the http2_server implementation, by replacing the current auth proxy implementation with a handler chain.

Changes

  • ...

thrau avatar Jun 04 '24 20:06 thrau

May not have been that easy after all. Will investigate further..

thrau avatar Jun 05 '24 00:06 thrau

I tried to re-write the entire proxying mechanism by re-using more from our handler chain, but there still seem to be several issues related to headers and streaming data, which I think is a result of the proxy mechanism on both sides doing too many things to the request.

I need to sit down with @whummer to better understand what the components should be doing exactly before moving forward, because I'm shotgun debugging at this point.

thrau avatar Jun 05 '24 20:06 thrau

Thanks for digging into this @thrau and getting the extension upgraded to the new gateway logic! 🙌 Happy to do some pairing on this - or maybe I'll find some time this weekend to dig into it a bit.. Let's compare notes soon!

whummer avatar Jun 08 '24 00:06 whummer

@thrau Sorry for the delay, finally got a chance to look into this. Great job on the refactoring and removing the use of http2_server! 🚀 Pushed a small change to drop Transfer-Encoding headers from the proxied response, see: https://github.com/localstack/localstack-extensions/pull/62/files#diff-ee721dbb335a260769d713b880c29dc772e825e268aedea2a0b3d67fd7f99bc3R46-R47

Was running some quick tests locally, seems to be working. 🎉 At least for S3 requests - these buckets are proxied from a real AWS account:

$ awslocal s3 ls
2024-06-20 22:25:33 ls-test-whummer-123
2024-06-20 22:51:04 test-bucket-a8c53ada

SQS tests still seem to be failing in the CI pipeline, we can look into those next.. 👀

whummer avatar Jun 20 '24 21:06 whummer