localstack
bug: API Gateway v2 HTTP_PROXY override path is not working
Is there an existing issue for this?
- [X] I have searched the existing issues
Current Behavior
I have the following construct created using cdklocal:
API GW v2 1 -> API GW v2 2 -> Lambda function.
API GW 1 has the following route:
awslocal apigatewayv2 get-routes --api-id 7d29c397
{
"Items": [
{
"ApiKeyRequired": false,
"AuthorizationType": "CUSTOM",
"AuthorizerId": "821dc6ce",
"RouteId": "61865a14",
"RouteKey": "ANY /es-batch/{proxy+}",
"Target": "c895c89a"
}
]
}
Here is a target for with route:
awslocal apigatewayv2 get-integrations --api-id 7d29c397
{
"Items": [
{
"ConnectionType": "INTERNET",
"Description": "Integration for es-batch module",
"IntegrationId": "c895c89a",
"IntegrationMethod": "ANY",
"IntegrationType": "HTTP_PROXY",
"IntegrationUri": "http://154f05a5.execute-api.amazonaws.com:4566/",
"PayloadFormatVersion": "2.0",
"RequestParameters": {
"overwrite:path": "${request.path}"
},
"TimeoutInMillis": 30000
}
]
}
API GW 2 has the following route:
awslocal apigatewayv2 get-routes --api-id 154f05a5
{
"Items": [
{
"AuthorizationType": "NONE",
"OperationName": "Proxy route",
"RouteId": "96b06ec3",
"RouteKey": "ANY /{proxy+}",
"Target": "eadfa35c"
}
]
}
The target for that route is the Lambda function. Here is how I can it using API GW 2:
curl --location 'http://154f05a5.execute-api.localhost.localstack.cloud:4566/es-batch/v1/data-pipelines?tenantId=tenant-local' --header 'Accept: application/json' response is 200.
However, when I use API GW 1 I receive 404 error:
curl --location 'http://7d29c397.execute-api.localhost.localstack.cloud:4566/es-batch/v1/data-pipelines?tenantId=tenant-local' --header 'Accept: application/json'
< HTTP/1.1 404
< Content-Type: text/plain; charset=utf-8
< Content-Length: 9
< Connection: close
< date: Tue, 09 Apr 2024 15:10:17 GMT
< server: hypercorn-h11
<
* Closing connection
Not Found
Here are the logs:
Start of execution environment 4faf06b69dc3b57cc4222f4242632aa0 for function arn:aws:lambda:us-east-1:000000000000:function:AuthorizerLambda:$LATEST took 758.54ms 2024-04-09 18:12:06 2024-04-09T15:12:06.858 DEBUG --- [ asgi_gw_1] l.s.l.i.docker_runtime_exe : Sending invoke-payload '{"invoke-id": "101e705b-133c-4e67-a446-51883c76212f", "invoked-function-arn": "arn:aws:lambda:us-east-1:000000000000:function:AuthorizerLambda", "payload": "{"version": "2.0", "type": "REQUEST", "routeArn": "arn:aws:execute-api:us-east-1:000000000000:7d29c397/$default/GET/es-batch/v1/data-pipelines", "identitySource": ["Bearer XXXX"], "routeKey": "ANY /es-batch/v1/data-pipelines", "rawPath": "/es-batch/v1/data-pipelines", "rawQueryString": "tenantId=tenant-local", "cookies": null, "headers": {"Accept": "application/json", "bd-host": "tenant-local.api.us1.bdconnected.com", "Authorization": "Bearer XXXX", "User-Agent": "PostmanRuntime/7.37.0", "Postman-Token": "6dc408cc-09a2-48bc-a410-2041b6ee6e5a", "Host": "7d29c397.execute-api.localhost.localstack.cloud:4566", "Accept-Encoding": "gzip, deflate, br", "Connection": "keep-alive", "X-Forwarded-For": "172.17.0.1, 7d29c397.execute-api.localhost.localstack.cloud:4566", "x-localstack-edge": "http://7d29c397.execute-api.localhost.localstack.cloud:4566"}, "queryStringParameters": {"tenantId": "tenant-local"}, "pathParameters": {"proxy": "v1/data-pipelines"}, "stageVariables": null, "requestContext": {"accountId": "000000000000", "apiId": "7d29c397", "authentication": null, "domainName": "7d29c397.execute-api.localhost.localstack.cloud", "...' to executor '4faf06b69dc3b57cc4222f4242632aa0' 2024-04-09 18:12:06 2024-04-09T15:12:06.862 INFO --- [ asgi_gw_2] localstack.request.http : POST /_localstack_lambda/4faf06b69dc3b57cc4222f4242632aa0/invocations/101e705b-133c-4e67-a446-51883c76212f/logs => 202 2024-04-09 18:12:06 2024-04-09T15:12:06.864 INFO --- [ asgi_gw_3] localstack.request.http : POST /localstack_lambda/4faf06b69dc3b57cc4222f4242632aa0/invocations/101e705b-133c-4e67-a446-51883c76212f/response => 202 2024-04-09 18:12:06 2024-04-09T15:12:06.878 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : Got logs for invocation '101e705b-133c-4e67-a446-51883c76212f' 2024-04-09 18:12:06 2024-04-09T15:12:06.878 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : [AuthorizerLambda-101e705b-133c-4e67-a446-51883c76212f] START RequestId: 101e705b-133c-4e67-a446-51883c76212f Version: $LATEST 2024-04-09 18:12:06 2024-04-09T15:12:06.880 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : [AuthorizerLambda-101e705b-133c-4e67-a446-51883c76212f] 2024/04/09 15:12:06 token: XXXX 2024-04-09 18:12:06 2024-04-09T15:12:06.880 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : [AuthorizerLambda-101e705b-133c-4e67-a446-51883c76212f] END RequestId: 101e705b-133c-4e67-a446-51883c76212f 2024-04-09 18:12:06 2024-04-09T15:12:06.880 DEBUG --- [ asgi_gw_1] l.s.l.i.version_manager : [AuthorizerLambda-101e705b-133c-4e67-a446-51883c76212f] REPORT RequestId: 101e705b-133c-4e67-a446-51883c76212f Duration: 1.35 ms Billed Duration: 2 ms Memory Size: 128 MB Max Memory Used: 128 MB 2024-04-09 18:12:06 2024-04-09T15:12:06.880 DEBUG --- [ asgi_gw_1] l.s.lambda.provider : Lambda invocation duration: 783.58ms 2024-04-09 18:12:06 2024-04-09T15:12:06.888 INFO --- [ asgi_gw_5] l.s.apigateway.authorizers : Received authorizer result: {"isAuthorized":true} 2024-04-09 18:12:06 2024-04-09T15:12:06.889 DEBUG --- [ asgi_gw_5] l.s.apigateway.authorizers : Updating identity context: {'cognitoIdentityId': None, 'cognitoIdentityPoolId': None} 2024-04-09 18:12:06 2024-04-09T15:12:06.899 DEBUG --- [ asgi_gw_5] l.s.a.integrations : Unable to extract message value from expression '${request.path}', falling back to using verbatim string: Error on line 1, col 1: Unexpected character: { 2024-04-09 18:12:06 2024-04-09T15:12:06.900 DEBUG --- [ asgi_gw_5] l.s.a.integrations : Sending GET request to http://154f05a5.execute-api.amazonaws.com:4566?tenantId=tenant-local in response to invocation message {'version': '2.0', 'routeKey': 'GET /es-batch/v1/data-pipelines', 'rawPath': '/es-batch/v1/data-pipelines', 'rawQueryString': 'tenantId=tenant-local', 'headers': {'accept': 'application/json', 'bd-host': 'tenant-local.api.us1.bdconnected.com', 'Authorization': 'Bearer XXXXXXXX', 'User-Agent': 'PostmanRuntime/7.37.0', 'Postman-Token': '6dc408cc-09a2-48bc-a410-2041b6ee6e5a', 'Host': '7d29c397.execute-api.localhost.localstack.cloud:4566', 'accept-encoding': 'gzip, deflate, br', 'Connection': 'keep-alive', 'x-localstack-edge': 'http://7d29c397.execute-api.localhost.localstack.cloud:4566', 'Accept': 'application/json', 'Accept-Encoding': 'gzip, deflate, br', 'Forwarded': 'by=172.17.0.1, 7d29c397.execute-api.localhost.localstack.cloud:4566'}, 'body': b'', 'requestContext': {'accountId': '4566/', 'apiId': '7d29c397', 'authorizer': {'lambda': {'isAuthorized': True}}, 'domainName': '7d29c397.execute-api.localhost.localstack.cloud', 'domainPrefix': '7d29c397', 'http': {'method': 'GET', 'path': '/es-batch/v1/data-pipelines', 'protocol': 'HTTP/1.1', 'sourceIp': '172.17.0.1', 'userAgent': 'PostmanRuntime/7.37.0'}, 'requestId': '39f9e5c6-3880-4398-befe-5c50148662da', 'routeKey': 'ANY /es-batch/v1/data-pipelines', 'stage': '$default', 'time': '09/Apr/2024:15:12:06 +0000', 'timeEpoch': 1712675526900}, 'isBase64Encoded': False, 'queryStringParameters': {'tenantId': 'tenant-local'}, 'stageVariables': {}, 'pathParameters': {'proxy': 'v1/data-pipelines'}} 2024-04-09 18:12:06 2024-04-09T15:12:06.908 DEBUG --- [ asgi_gw_3] l.s.a.apigateway_utils : No matching stage (None) for invocation path (falling back to $default stage): / 2024-04-09 18:12:06 2024-04-09T15:12:06.908 ERROR --- [ asgi_gw_3] l.s.a.apigateway_utils : Unable to find a matching route for GET / 2024-04-09 18:12:06 2024-04-09T15:12:06.908 DEBUG --- [ asgi_gw_3] l.s.apigateway.authorizers : No authorizer configured for resource: GET /?tenantId=tenant-local 2024-04-09 18:12:06 2024-04-09T15:12:06.909 DEBUG --- [ asgi_gw_3] l.s.apigateway.authorizers : Authorization failed: Not Found - Traceback (most recent call last): 2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/authorizers.py.enc", line 345, in is_request_authorized 2024-04-09 18:12:06 try:AuthorizerService().check_request_authorization(invocation_context) 2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/authorizers.py.enc", line 315, in check_request_authorization 2024-04-09 18:12:06 try:J=get_target_resource_method_or_route(A) 2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/apigateway_utils.py.enc", line 228, in get_target_resource_method_or_route 2024-04-09 18:12:06 return find_matching_route(A) 2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^ 2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/apigateway_utils.py.enc", line 201, in find_matching_route 2024-04-09 18:12:06 else:return find_matching_http_route(B,A) 2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-04-09 18:12:06 File "/opt/code/localstack/.venv/lib/python3.11/site-packages/localstack_ext/services/apigateway/apigateway_utils.py.enc", line 193, in find_matching_http_route 2024-04-09 18:12:06 if not A:LOG.error(f"Unable to find a matching route for {F} {E}");raise NotFoundException(_N) 2024-04-09 18:12:06 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-04-09 18:12:06 localstack_ext.aws.api.apigatewayv2.NotFoundException: Not Found 2024-04-09 18:12:06 2024-04-09 18:12:06 2024-04-09T15:12:06.910 INFO --- [ asgi_gw_3] localstack.request.http : GET / => 404
To summarize:
It seems "overwrite:path": "${request.path}" does not work in Localstack as the same CDK code works in AWS account.
Expected Behavior
API GW 1 integration should replace the path in the request to API GW 2 with the original path using next configuration of the HTTP_PROXY integration: "overwrite:path": "${request.path}"
How are you starting LocalStack?
With a docker-compose file
Steps To Reproduce
How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)
localstack start -d
Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)
cio, err := input.ApigwClient.CreateIntegration(ctx, &apigatewayv2.CreateIntegrationInput{
ApiId: awsClient.String(input.APIGatewayID),
IntegrationType: gwTypes.IntegrationTypeHttpProxy,
ConnectionType: gwTypes.ConnectionTypeInternet,
Description: awsClient.String(fmt.Sprintf("Integration for %s module", input.Slug)),
IntegrationMethod: awsClient.String("ANY"),
IntegrationUri: awsClient.String(input.InternalAPIGatewayURL),
PayloadFormatVersion: awsClient.String("1.0"),
RequestParameters: map[string]string{
"overwrite:path": "$request.path",
},
}, func(_ *apigatewayv2.Options) {})
if err != nil {
return nil, err //nolint:wrapcheck // no need to wrap here
}
authorizationType := gwTypes.AuthorizationTypeNone
if input.AuthorizerID != "" {
authorizationType = gwTypes.AuthorizationTypeCustom
}
route, err := input.ApigwClient.CreateRoute(ctx, &apigatewayv2.CreateRouteInput{
ApiId: awsClient.String(input.APIGatewayID),
RouteKey: awsClient.String(fmt.Sprintf("%s %s", input.Method, input.Route)),
AuthorizationType: authorizationType,
ApiKeyRequired: optional.Option(false),
AuthorizerId: optional.Option(input.AuthorizerID),
Target: awsClient.String(fmt.Sprintf("integrations/%s", *cio.IntegrationId)),
}, func(_ *apigatewayv2.Options) {})
if err != nil {
return nil, err //nolint:wrapcheck // no need to wrap here
}
Environment
- OS: OSX 14.4.1
- LocalStack: 3.3.0 Pro
Anything else?
Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our Slack Community for real-time help and drop a message to LocalStack Pro Support if you are a Pro user! If you are willing to contribute towards fixing this issue, please have a look at our contributing guidelines and our contributing guide.
Hello, I am evaluating pro version to use http-proxy with apigateway-v2.
I am running into this same issue. Is there any update on this?
When I tried using the following requestParameter, I notice that the value of request.path.xyz comes up as empty.
for path like "GET /portal/{xyz+}"
"overwrite:path" = "/$request.path.xyz"
Note that when I use a static string, the overwrite functionality works fine.
for path like "GET /portal/{xyz+}"
"overwrite:path" = "/static/path"
To compare this with REST/awsgateway-v1, I am able to get this to work with the following. I do not know if there is something similar in http/awsgateway-v2 that I can use.
"integration.request.path.xyz" = "method.request.path.xyz"
Hello @MrZoidberg and @a26-beep, and thanks for your report!
This issue should now be fixed by using the following config flag: PROVIDER_OVERRIDE_APIGATEWAY=next_gen, starting with LocalStack 3.8. You can read more about it in our documentation: https://docs.localstack.cloud/user-guide/aws/apigateway/#new-api-gateway-implementation
This should hopefully fix your issue. Thank you!
That's cool. I'm no longer on the project where I need this functionality. I did a quick check and it seems to be working fine.
Thanks for the feedback @MrZoidberg! Sorry it took so long 😅 I'll close the issue then. Thanks again for the report 🙏