localstack
localstack copied to clipboard
localstack
bug: Failed to create RDS cluster on `latest`
Is there an existing issue for this?
- [X] I have searched the existing issues
Current Behavior
When sending createRDSCluster with the following configuration:
createDbClusterRequest <-
pure $
AWS.newCreateDBCluster dbClusterId "aurora-postgresql"
& the @"masterUsername" .~ Just masterUsername
& the @"masterUserPassword" .~ Just masterPassword
& the @"enableHttpEndpoint" .~ Just True
& the @"databaseName" .~ Just databaseName
the request fails with:
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Trace, builder = "[Version 4 Metadata] {\n time = 2024-09-22 12:49:45.715741792 UTC\n endpoint = localhost\n credential = test/20240922/us-east-1/rds/aws4_request\n signed headers = content-type;host;x-amz-content-sha256;x-amz-date\n signature = 0addc26408ddc79a4ce0f655d77d4beef389c1082f618ff03b194648728188dd\n string to sign = {\nAWS4-HMAC-SHA256\n20240922T124945Z\n20240922/us-east-1/rds/aws4_request\nf3fa54eb85f12e8023ce437613d8c92f0d5ad5b2098cab55cba44edf5ce0d8d3\n }\n canonical request = {\nPOST\n/\n\ncontent-type:application/x-www-form-urlencoded; charset=utf-8\nhost:localhost:32779\nx-amz-content-sha256:3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\nx-amz-date:20240922T124945Z\n\ncontent-type;host;x-amz-content-sha256;x-amz-date\n3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\n }\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[Client Request] {\n host = localhost:32779\n secure = False\n method = POST\n target = Nothing\n timeout = ResponseTimeoutMicro 70000000\n redirects = 0\n path = /\n query = \n headers = x-amz-content-sha256: 3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de; x-amz-date: 20240922T124945Z; host: localhost:32779; content-type: application/x-www-form-urlencoded; charset=utf-8; authorization: AWS4-HMAC-SHA256 Credential=test/20240922/us-east-1/rds/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=0addc26408ddc79a4ce0f655d77d4beef389c1082f618ff03b194648728188dd\n body = Action=CreateDBCluster&DBClusterIdentifier=my-cluster&DatabaseName=rds_data_migration&EnableHttpEndpoint=true&Engine=aurora-postgresql&MasterUserPassword=masterPassword&MasterUsername=masterUsername&Version=2014-10-31\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[HttpException] {\nHttpExceptionRequest Request {\n host = \"localhost\"\n port = 32779\n secure = False\n requestHeaders = [(\"X-Amz-Content-SHA256\",\"3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\"),(\"X-Amz-Date\",\"20240922T124945Z\"),(\"Host\",\"localhost:32779\"),(\"Content-Type\",\"application/x-www-form-urlencoded; charset=utf-8\"),(\"Authorization\",\"<REDACTED>\")]\n path = \"/\"\n queryString = \"\"\n method = \"POST\"\n proxy = Nothing\n rawBody = False\n redirectCount = 0\n responseTimeout = ResponseTimeoutMicro 70000000\n requestVersion = HTTP/1.1\n proxySecureMode = ProxySecureWithConnect\n}\n (InternalException Network.Socket.recvBuf: resource vanished (Connection reset by peer))\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[Retry http_error] after 1 attempt(s)."}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Trace, builder = "[Version 4 Metadata] {\n time = 2024-09-22 12:49:45.741221611 UTC\n endpoint = localhost\n credential = test/20240922/us-east-1/rds/aws4_request\n signed headers = content-type;host;x-amz-content-sha256;x-amz-date\n signature = 0addc26408ddc79a4ce0f655d77d4beef389c1082f618ff03b194648728188dd\n string to sign = {\nAWS4-HMAC-SHA256\n20240922T124945Z\n20240922/us-east-1/rds/aws4_request\nf3fa54eb85f12e8023ce437613d8c92f0d5ad5b2098cab55cba44edf5ce0d8d3\n }\n canonical request = {\nPOST\n/\n\ncontent-type:application/x-www-form-urlencoded; charset=utf-8\nhost:localhost:32779\nx-amz-content-sha256:3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\nx-amz-date:20240922T124945Z\n\ncontent-type;host;x-amz-content-sha256;x-amz-date\n3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\n }\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[Client Request] {\n host = localhost:32779\n secure = False\n method = POST\n target = Nothing\n timeout = ResponseTimeoutMicro 70000000\n redirects = 0\n path = /\n query = \n headers = x-amz-content-sha256: 3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de; x-amz-date: 20240922T124945Z; host: localhost:32779; content-type: application/x-www-form-urlencoded; charset=utf-8; authorization: AWS4-HMAC-SHA256 Credential=test/20240922/us-east-1/rds/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=0addc26408ddc79a4ce0f655d77d4beef389c1082f618ff03b194648728188dd\n body = Action=CreateDBCluster&DBClusterIdentifier=my-cluster&DatabaseName=rds_data_migration&EnableHttpEndpoint=true&Engine=aurora-postgresql&MasterUserPassword=masterPassword&MasterUsername=masterUsername&Version=2014-10-31\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[HttpException] {\nHttpExceptionRequest Request {\n host = \"localhost\"\n port = 32779\n secure = False\n requestHeaders = [(\"X-Amz-Content-SHA256\",\"3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\"),(\"X-Amz-Date\",\"20240922T124945Z\"),(\"Host\",\"localhost:32779\"),(\"Content-Type\",\"application/x-www-form-urlencoded; charset=utf-8\"),(\"Authorization\",\"<REDACTED>\")]\n path = \"/\"\n queryString = \"\"\n method = \"POST\"\n proxy = Nothing\n rawBody = False\n redirectCount = 0\n responseTimeout = ResponseTimeoutMicro 70000000\n requestVersion = HTTP/1.1\n proxySecureMode = ProxySecureWithConnect\n}\n NoResponseDataReceived\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[Retry http_error] after 2 attempt(s)."}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Trace, builder = "[Version 4 Metadata] {\n time = 2024-09-22 12:49:45.791977927 UTC\n endpoint = localhost\n credential = test/20240922/us-east-1/rds/aws4_request\n signed headers = content-type;host;x-amz-content-sha256;x-amz-date\n signature = 0addc26408ddc79a4ce0f655d77d4beef389c1082f618ff03b194648728188dd\n string to sign = {\nAWS4-HMAC-SHA256\n20240922T124945Z\n20240922/us-east-1/rds/aws4_request\nf3fa54eb85f12e8023ce437613d8c92f0d5ad5b2098cab55cba44edf5ce0d8d3\n }\n canonical request = {\nPOST\n/\n\ncontent-type:application/x-www-form-urlencoded; charset=utf-8\nhost:localhost:32779\nx-amz-content-sha256:3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\nx-amz-date:20240922T124945Z\n\ncontent-type;host;x-amz-content-sha256;x-amz-date\n3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\n }\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[Client Request] {\n host = localhost:32779\n secure = False\n method = POST\n target = Nothing\n timeout = ResponseTimeoutMicro 70000000\n redirects = 0\n path = /\n query = \n headers = x-amz-content-sha256: 3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de; x-amz-date: 20240922T124945Z; host: localhost:32779; content-type: application/x-www-form-urlencoded; charset=utf-8; authorization: AWS4-HMAC-SHA256 Credential=test/20240922/us-east-1/rds/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=0addc26408ddc79a4ce0f655d77d4beef389c1082f618ff03b194648728188dd\n body = Action=CreateDBCluster&DBClusterIdentifier=my-cluster&DatabaseName=rds_data_migration&EnableHttpEndpoint=true&Engine=aurora-postgresql&MasterUserPassword=masterPassword&MasterUsername=masterUsername&Version=2014-10-31\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[HttpException] {\nHttpExceptionRequest Request {\n host = \"localhost\"\n port = 32779\n secure = False\n requestHeaders = [(\"X-Amz-Content-SHA256\",\"3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\"),(\"X-Amz-Date\",\"20240922T124945Z\"),(\"Host\",\"localhost:32779\"),(\"Content-Type\",\"application/x-www-form-urlencoded; charset=utf-8\"),(\"Authorization\",\"<REDACTED>\")]\n path = \"/\"\n queryString = \"\"\n method = \"POST\"\n proxy = Nothing\n rawBody = False\n redirectCount = 0\n responseTimeout = ResponseTimeoutMicro 70000000\n requestVersion = HTTP/1.1\n proxySecureMode = ProxySecureWithConnect\n}\n NoResponseDataReceived\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[Retry http_error] after 3 attempt(s)."}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Trace, builder = "[Version 4 Metadata] {\n time = 2024-09-22 12:49:45.892710367 UTC\n endpoint = localhost\n credential = test/20240922/us-east-1/rds/aws4_request\n signed headers = content-type;host;x-amz-content-sha256;x-amz-date\n signature = 0addc26408ddc79a4ce0f655d77d4beef389c1082f618ff03b194648728188dd\n string to sign = {\nAWS4-HMAC-SHA256\n20240922T124945Z\n20240922/us-east-1/rds/aws4_request\nf3fa54eb85f12e8023ce437613d8c92f0d5ad5b2098cab55cba44edf5ce0d8d3\n }\n canonical request = {\nPOST\n/\n\ncontent-type:application/x-www-form-urlencoded; charset=utf-8\nhost:localhost:32779\nx-amz-content-sha256:3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\nx-amz-date:20240922T124945Z\n\ncontent-type;host;x-amz-content-sha256;x-amz-date\n3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\n }\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[Client Request] {\n host = localhost:32779\n secure = False\n method = POST\n target = Nothing\n timeout = ResponseTimeoutMicro 70000000\n redirects = 0\n path = /\n query = \n headers = x-amz-content-sha256: 3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de; x-amz-date: 20240922T124945Z; host: localhost:32779; content-type: application/x-www-form-urlencoded; charset=utf-8; authorization: AWS4-HMAC-SHA256 Credential=test/20240922/us-east-1/rds/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=0addc26408ddc79a4ce0f655d77d4beef389c1082f618ff03b194648728188dd\n body = Action=CreateDBCluster&DBClusterIdentifier=my-cluster&DatabaseName=rds_data_migration&EnableHttpEndpoint=true&Engine=aurora-postgresql&MasterUserPassword=masterPassword&MasterUsername=masterUsername&Version=2014-10-31\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Debug, builder = "[HttpException] {\nHttpExceptionRequest Request {\n host = \"localhost\"\n port = 32779\n secure = False\n requestHeaders = [(\"X-Amz-Content-SHA256\",\"3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\"),(\"X-Amz-Date\",\"20240922T124945Z\"),(\"Host\",\"localhost:32779\"),(\"Content-Type\",\"application/x-www-form-urlencoded; charset=utf-8\"),(\"Authorization\",\"<REDACTED>\")]\n path = \"/\"\n queryString = \"\"\n method = \"POST\"\n proxy = Nothing\n rawBody = False\n redirectCount = 0\n responseTimeout = ResponseTimeoutMicro 70000000\n requestVersion = HTTP/1.1\n proxySecureMode = ProxySecureWithConnect\n}\n NoResponseDataReceived\n}"}
┃ │ AwsLogEntry {callStack = [("createRdsDbCluster",SrcLoc {srcLocPackage = "rds-data-0.0.0.7-inplace-rds-data-integration", srcLocModule = "Test.Data.RdsData.Migration.ConnectionSpec", srcLocFile = "integration/Test/Data/RdsData/Migration/ConnectionSpec.hs", srcLocStartLine = 43, srcLocStartCol = 28, srcLocEndLine = 43, srcLocEndCol = 46})], logLevel = Error, builder = "[HttpException] {\nHttpExceptionRequest Request {\n host = \"localhost\"\n port = 32779\n secure = False\n requestHeaders = [(\"X-Amz-Content-SHA256\",\"3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de\"),(\"X-Amz-Date\",\"20240922T124945Z\"),(\"Host\",\"localhost:32779\"),(\"Content-Type\",\"application/x-www-form-urlencoded; charset=utf-8\"),(\"Authorization\",\"<REDACTED>\")]\n path = \"/\"\n queryString = \"\"\n method = \"POST\"\n proxy = Nothing\n rawBody = False\n redirectCount = 0\n responseTimeout = ResponseTimeoutMicro 70000000\n requestVersion = HTTP/1.1\n proxySecureMode = ProxySecureWithConnect\n}\n NoResponseDataReceived\n}"}
┃ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
┃ │ TransportError (HttpExceptionRequest Request {
┃ │ host = "localhost"
┃ │ port = 32779
┃ │ secure = False
┃ │ requestHeaders = [("X-Amz-Content-SHA256","3308b32322c38778a429bb746e67d1192a6c3dab0a35c83f600cb68c9b6794de"),("X-Amz-Date","20240922T124945Z"),("Host","localhost:32779"),("Content-Type","application/x-www-form-urlencoded; charset=utf-8"),("Authorization","<REDACTED>")]
┃ │ path = "/"
┃ │ queryString = ""
┃ │ method = "POST"
┃ │ proxy = Nothing
┃ │ rawBody = False
┃ │ redirectCount = 0
┃ │ responseTimeout = ResponseTimeoutMicro 70000000
┃ │ requestVersion = HTTP/1.1
┃ │ proxySecureMode = ProxySecureWithConnect
┃ │ }
┃ │ NoResponseDataReceived)
Expected Behavior
The request should succeed.
The last known working version is localstack/localstack-pro:latest
How are you starting LocalStack?
With a docker-compose file
Steps To Reproduce
How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)
docker run localstack/localstack
Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)
N/A
Environment
- OS: Linux
- LocalStack:
LocalStack version: `localstack/localstack-pro:latest`
LocalStack Docker image sha: `7eaad79a9783`
LocalStack build date:
LocalStack build git hash:
Anything else?
No response
Hi @newhoggy,
could you please also set DEBUG=1 when starting LocalStack and provide the localstack logs when you run this request? 🙏
With the output you provided I can only verify that you received Timeouts for the requests, but it's not clear if the requests were actually handled by LocalStack.
Hmm, you might be onto something. I don't see localstack starting at all. Why would it be the case when I use latest the container doesn't start.
Oh, I spoke too soon. I see it now. I think it timed out downloading the image on the first attempt.
I got a good run:
jky@Terminus:~$ docker logs -f 89fb58df2668
LocalStack version: 3.7.3.dev40
LocalStack build date: 2024-09-23
LocalStack build git hash: 550f4f95
Docker not available
2024-09-23T16:22:38.808 INFO --- [ MainThread] l.p.c.b.licensingv2 : Successfully requested and activated new license 123d4cdb-7b40-49c6-97ee-ea7fe1243cf8:hobby 🔑✅
2024-09-23T16:22:42.062 INFO --- [ MainThread] l.p.c.extensions.platform : loaded 0 extensions
Ready.
2024-09-23T16:22:46.011 INFO --- [ Thread-7] l.p.core.utils.postgresql : Starting PostgreSQL query listener/rewriter on port 4510->36507
2024-09-23T16:22:47.020 WARN --- [-functhread5] l.pro.core.packages.core : postgresql will be installed as an OS package, even though install target is _not_ set to be static.
2024-09-23T16:22:53.011 INFO --- [et.reactor-0] localstack.request.aws : AWS rds.CreateDBCluster => 200
2024-09-23T16:22:53.021 INFO --- [et.reactor-0] localstack.request.aws : AWS secretsmanager.CreateSecret => 200
2024-09-23T16:22:53.063 INFO --- [ Thread-17] l.p.core.utils.postgresql : Starting PostgreSQL query listener/rewriter on port 4511->46489
2024-09-23T16:23:00.065 INFO --- [et.reactor-1] localstack.request.aws : AWS rds.CreateDBInstance => 200
2024-09-23T16:23:00.068 INFO --- [et.reactor-1] localstack.request.aws : AWS rds.DescribeDBClusters => 200
2024-09-23T16:23:01.071 INFO --- [et.reactor-0] localstack.request.aws : AWS rds.DescribeDBClusters => 200
2024-09-23T16:23:02.073 INFO --- [et.reactor-1] localstack.request.aws : AWS rds.DescribeDBClusters => 200
2024-09-23T16:23:03.076 INFO --- [et.reactor-0] localstack.request.aws : AWS rds.DescribeDBClusters => 200
2024-09-23T16:23:04.080 INFO --- [et.reactor-1] localstack.request.aws : AWS rds.DescribeDBClusters => 200
2024-09-23T16:23:04.890 INFO --- [-functhread5] l.p.c.s.r.engine_postgres : Starting RDS server on port 4510 (backend port 36507) - database "rds_data_migration", user "masterUsername" - data dir /var/lib/localstack/tmp/rds/postgres/000000000000/cluster/000000000000/us-east-1/my-cluster
2024-09-23T16:23:04.890 INFO --- [-functhread6] l.p.c.s.r.engine_postgres : Starting RDS server on port 4511 (backend port 46489) - database "test", user "test" - data dir /var/lib/localstack/tmp/rds/postgres/000000000000/instance/000000000000/us-east-1/my-cluster
2024-09-23T16:23:05.082 INFO --- [et.reactor-0] localstack.request.aws : AWS rds.DescribeDBClusters => 200
2024-09-23T16:23:06.086 INFO --- [et.reactor-1] localstack.request.aws : AWS rds.DescribeDBClusters => 200
2024-09-23T16:23:07.090 INFO --- [et.reactor-0] localstack.request.aws : AWS rds.DescribeDBClusters => 200
2024-09-23T16:23:08.094 INFO --- [et.reactor-0] localstack.request.aws : AWS rds.DescribeDBClusters => 200
2024-09-23T16:23:08.126 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'CREATE TABLE IF NOT EXISTS migration ( ulid CHAR(26) NOT NULL PRIMARY KEY, created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, deployed_by TEXT NOT NULL, CONSTRAINT valid_ulid_constraint CHECK (ulid::text ~ '^[0-9A-HJKMNP-TV-Z]{26}$'))'
2024-09-23T16:23:08.151 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.153 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'CREATE INDEX IF NOT EXISTS idx_migration_created_at ON migration (created_at)'
2024-09-23T16:23:08.164 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.165 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'CREATE INDEX IF NOT EXISTS idx_migration_deployed_by ON migration (deployed_by)'
2024-09-23T16:23:08.177 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.179 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'CREATE TABLE users (name VARCHAR(50))'
2024-09-23T16:23:08.190 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.192 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'CREATE INDEX idx_users_name ON users (name)'
2024-09-23T16:23:08.203 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.204 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'CREATE TABLE projects (name VARCHAR(50))'
2024-09-23T16:23:08.216 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.217 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'CREATE INDEX idx_projects_name ON projects (name)'
2024-09-23T16:23:08.228 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.230 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'INSERT INTO users (name) VALUES ('Alice')'
2024-09-23T16:23:08.241 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.242 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'SELECT COUNT(name) FROM users'
2024-09-23T16:23:08.254 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.255 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'SELECT table_name
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_type = 'BASE TABLE''
2024-09-23T16:23:08.268 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.270 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'SELECT schema_name
FROM information_schema.schemata'
2024-09-23T16:23:08.282 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.283 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'CREATE TABLE examples (ulid CHAR(26), example_data BYTEA NOT NULL, example_hash CHAR(64) UNIQUE, PRIMARY KEY ( ulid, example_hash ), CONSTRAINT valid_ulid_constraint CHECK ( ulid ~ '^[0-9A-HJKMNP-TV-Z]{26}' ))'
2024-09-23T16:23:08.298 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.299 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'SELECT table_name FROM information_schema.tables WHERE table_schema = 'public' AND table_type = 'BASE TABLE''
2024-09-23T16:23:08.313 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.326 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.328 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'DROP INDEX idx_projects_name'
2024-09-23T16:23:08.338 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.339 WARN --- [et.reactor-1] l.p.c.s.rds_data.provider : ExecuteStatement can only execute one statement at a time, executing only 'DROP TABLE projects'
2024-09-23T16:23:08.351 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.363 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.375 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
2024-09-23T16:23:08.390 INFO --- [et.reactor-1] localstack.request.aws : AWS rds-data.ExecuteStatement => 200
This it what it looks like for the bad run:
jky@Terminus:~$ docker logs -f d8a01236c386
LocalStack version: 3.7.3.dev40
LocalStack build date: 2024-09-23
LocalStack build git hash: 550f4f95
Docker not available
2024-09-23T16:24:50.891 INFO --- [ MainThread] l.p.c.b.licensingv2 : Successfully requested and activated new license 123d4cdb-7b40-49c6-97ee-ea7fe1243cf8:hobby 🔑✅
2024-09-23T16:24:54.139 INFO --- [ MainThread] l.p.c.extensions.platform : loaded 0 extensions
Ready.
Okay, I suspect it is because I am connecting to the localstack port too quickly after starting the container and the port isn't ready yet. It must be taking slightly long to get the port running in the newer versions of the container.
If I add a 1 second sleep after container launch, it works more reliably.
You could the internal health-endpoint or the init-hooks to make sure that LocalStack is ready before sending requests.
In case you use the localstack cli, you could also use the waitcommand, e.g.
localstack start -d
localstack wait
Yeah, I can do that. I'll put in a little sleep for now and add the health endpoint check later. Happy to close.