Cannot use SDK v3 with dynamodb local - nodejs v18

[Berger92]

Checkboxes for prior research

• [X] I've gone through Developer Guide and API reference
• [X] I've checked AWS Forums and StackOverflow.
• [X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

Hi 👋

I've seen this issue in the past, but they have all been closed. I am migrating from my app using SDK v2 to SDK v3.

I'm also updating nodejs version to 18, but it doesn't seem to work.

I've tried tinkering with config, adding fake credentials, etc. but I got nowhere.

If I use nodejs 16, it does work.

SDK version number

v3.231.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

18.12.1

Reproduction Steps

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocument } from '@aws-sdk/lib-dynamodb';

const client = new DynamoDBClient({
  endpoint: 'http://localhost:8000'
});
const ddbDocClient = DynamoDBDocument.from(client);

(async () => {
  const result = await client.send(new ListTablesCommand({}));

  // or const result = await ddbDocClient.send(new ListTablesCommand({}));

  console.log(result);
})();

Observed Behavior

Error: connect ECONNREFUSED ::1:8000
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1300:16) {
  errno: -61,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '::1',
  port: 8000,
  '$metadata': { attempts: 1, totalRetryDelay: 0 }
}

Expected Behavior

The list of local dynamodb tables

Possible Solution

No response

Additional Information/Context

No response

[yenfryherrerafeliz]

Hi @Berger92, thanks for opening this issue. Could you please try specifying the endpoint as follow:

const client = new DynamoDBClient({
    endpoint: {
        hostname: 'localhost',
        port: 8000,
        path: '',
        protocol: 'http:',
    }
});

Your example code should look as follow:

import {DynamoDBClient, ListTablesCommand} from '@aws-sdk/client-dynamodb';
import { DynamoDBDocument } from '@aws-sdk/lib-dynamodb';

const client = new DynamoDBClient({
    endpoint: {
        hostname: 'localhost',
        port: 8000,
        path: '',
        protocol: 'http:',
    }
});
const ddbDocClient = DynamoDBDocument.from(client);

(async () => {
    const result = await ddbDocClient.send(new ListTablesCommand({}));

    console.log(result);
})();

Let me know if that works.

Thanks!

[Berger92]

Hi @yenfryherrerafeliz 👋 Thanks for the quick response. I've tried it, but I got the same result as before

Error: connect ECONNREFUSED ::1:8000
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1300:16) {
  errno: -61,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '::1',
  port: 8000,
  '$metadata': { attempts: 1, totalRetryDelay: 0 }
}

[Berger92]

@yenfryherrerafeliz Just one additional piece of information that might help. I am using DynamoDB Local in a Docker container, latest version.

[yenfryherrerafeliz]

@Berger92, thanks for the information. This also helps with my reproduction, since in my local environment "localhost" is being resolved as 127.0.0.1. I will try with this new information and I will get back to you as soon as possible.

Thanks!

[yenfryherrerafeliz]

Hi @Berger92, sorry for the delay in getting back. After doing some researches I found that the issue that you are getting may be due to a factor out of the SDK itself. According with the error that you are getting, seems like the connection to the specified host and port is being refused, and this could be due to different situations, maybe the specified port is not open to be accessible, out of the container, and actually did you make sure you are running the image with the right port mapping?, because it seems to be a networking issue.

Thanks!

[Berger92]

Hi @yenfryherrerafeliz . I disagree with you, because as I said in the original post, the docker container works for me perfectly with SDK v3 using Nodejs16. The problem started occurring using Nodejs 18. (In addition, the aws cli also works)

[alexhddev]

Any updates? Having the same issue using the VSCode debugger. Update: The same issue appeared for my because I had a typo when I configured the AWS_REGION.

[codyfyi]

This may be resolved when #4466 is fixed.

In my case I'm not using docker, but I was getting this same error because I was sending the dynamodb command too fast after spinning up a local dynamodb instance. I found that if I put a 1 second pause in before the ddbDocClient.send(...), my problem went away.

[evanjd]

I encountered this issue as well. It ended up being a combination of a couple issues (unrelated to the AWS SDK):

• Node 17+ will now resolve to the 1st IP address returned by your DNS resolver, instead of always favouring IPv4 addresses: https://github.com/nodejs/node/pull/39987
• I was running dynamodb-local from docker, and my docker runtime did not bind to my IPv6 interface: https://github.com/abiosoft/colima/issues/583

Changing localhost to 127.0.0.1 resolved the issue for me.

[sonjz]

I concur using 127.0.0.1 instead of localhost (which seems to resolve to ::1), works with dynamodb local. figuring it doesn't like the ::1 loopback notation.

using this without issue:

config = {
  endpoint: "http://127.0.0.1:8123",
  region: "localhost"
};

dynamodb = new DynamoDBClient(config);

[throrin19]

Same problem with SQS part and use the same resolution : with localhost, it does not works but with 127.0.0.1 it works fine....

[IevgenRagulin]

Switching to 127.0.0.1 didn't do the trick for me. What did help though is providing dummy credentials to the dynamo client like this:

    const client = new DynamoDBClient({
        endpoint: `http://localhost:8000`,
        region: 'local',
        credentials: {
            accessKeyId: 'dummy',
            secretAccessKey: 'dummy'
        }
    })

Found the solution here: https://stackoverflow.com/questions/43322536/could-not-load-credentials-from-any-providers-while-using-dynamodb-locally-in

My issue was different though than described in this question. It seemed that SDK simply ignored the endpoint argument in my case, and kept trying to reach out to the real AWS services, I was getting this error:

Could not load credentials from any providers
CredentialsProviderError: Could not load credentials from any providers

[MateuszKikmunter]

Switching to 127.0.0.1 didn't do the trick for me. What did help though is providing dummy credentials to the dynamo client like this:

    const client = new DynamoDBClient({
        endpoint: `http://localhost:8000`,
        region: 'local',
        credentials: {
            accessKeyId: 'dummy',
            secretAccessKey: 'dummy'
        }
    })

Found the solution here: https://stackoverflow.com/questions/43322536/could-not-load-credentials-from-any-providers-while-using-dynamodb-locally-in

My issue was different though. It seemed that SDK simply ignored the endpoint argument in my case, and kept trying to reach out to the real AWS services, I was getting this error:

Could not load credentials from any providers
CredentialsProviderError: Could not load credentials from any providers

Thanks, I had the same issue and your solution worked for me as well! :)

[xiaohk]

I'm facing the same issue. Changing localhost to 127.0.0.1 doesn't fix it. Adding dummy credentials doesn't help either. I'm using the local dynamodb from NoSQL Workbench.

aws dynamodb list-tables --endpoint-url http://localhost:8029 works fine.

My error code is a bit different though.

{
  errno: -111,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 8029,
  '$metadata': { attempts: 3, totalRetryDelay: 90 }
}

[justsml]

This little 'subtle' change has cost me so much time 😭

Instead of 127.0.0.1, 👎 Use 0.0.0.0 👍

❓127.0.0.* will always be exclusively local - making some things more difficult. (e.g. seamless interop with docker, clear ipv4 behavior)

✅ 0.0.0.0 will allow both access and binding (to the host) using IPv4.

const client = new DynamoDBClient({
  endpoint: `http://0.0.0.0:8000`
});

ProTip: You can even use http://0.0.0.0:8000 in your browser for local web sites. 🚀

(Note: When binding services to 0.0.0.0 it could make them accessible to anyone on your WiFi/local network. Use best practices, judgement, firewall, docker networks, safe WiFi, etc.)

[shellscape]

The soultion proposed by @justsml isn't working for us:

Error: connect ECONNREFUSED 0.0.0.0:65337
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) {
  errno: -61,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '0.0.0.0',
  port: 65337,
  '$metadata': { attempts: 3, totalRetryDelay: 57 }
}

Neither is the 127.0.0.1 solution.

[cameron-tabula]

I can confirm that the following is working for me at least, using DynamoDB Local running from NoSQL Workbench, running on port 8111 (instead of the default 8000).

import { DynamoDBClient } from "@aws-sdk/client-dynamodb";

const client = new DynamoDBClient({
    endpoint: {
        // region: 'local',  // can be omitted
        // credentials: {accessKeyId: 'dummy', secretAccessKey: 'dummy'}, // use ~/.aws/credentials etc. 
        hostname: 'localhost',
        port: 8111,
        path: '',
        protocol: 'http:',
        // http://localhost:8111'
    },
});

// This ALSO works, and is likely what you really want (found it while searching for 'Local' in the SDK)

const client = new DynamoDBClient({
    endpoint: {
        url: 'http://localhost:8111'    // <-- just like --endpoint-url
    },
});

// not shown, but I used a ScanCommand after this to verify it was actually usable.

Things to note:

• You'll likely want to use the {endpoint: {url: ...}} construction rather than the expanded form. (this doesn't appear in any documentation I could find)
• The 'protocol' needs to have a colon included... if you were to use a URL parsing library I would expect that to be trimmed off, as the colon is not really part of the service.
• Both 'region' and 'credentials' can be omitted; ~/.aws/credentials etc. will be used. (presumably AWS_PROFILE could also be used, but I haven't tested that)

If you are connecting to Docker and getting ECONNREFUSED, test that the connectivity is working outside of Node.js... you might need to something like one or more of the following:

• Expose the port so it can be connected to from the host.
• Use 'host' network driver
• If your code is running inside another container (eg. using VSCode DevContainers) then not using 'localhost' or '127.0.0.1', but instead the name of the container where DynamoDB Local is running, OR something like host.docker.internal to reach the host on Docker Desktop.
• Adjust firewall policies (hopefully not, but I have seen this be a problem in the past even for local development with tools like Docker or Vagrant on Windows with centrally managed firewall policies)
• Other more advance security policies could be getting in the way also.

In short, make sure you try and test connectivity with a tool such as 'nc' (alternatively 'telnet' or PuTTY can be used to test connectivity to any (raw) port without expecting an actual TTY)

Some useful diagnostic commands (Linux)

$ sudo lsof -Pni:8111
COMMAND    PID  USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
java    315379 ckerr  197u  IPv6 7511992      0t0  TCP *:8111 (LISTEN)

$ ps -p 315379 -o command
COMMAND
java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -port 8111 -sharedDb -dbPath /home/ckerr/.aws/workbench

$ nc -zv 127.0.0.1 8111
Connection to 127.0.0.1 8111 port [tcp/*] succeeded!
$ nc -zv localhost 8111
Connection to localhost (127.0.0.1) 8111 port [tcp/*] succeeded!
$ nc -zv ::1 8111
Connection to ::1 8111 port [tcp/*] succeeded!
$ nc -zv :: 8111
Connection to :: 8111 port [tcp/*] succeeded!
$ nc -zv 0.0.0.0 8111
Connection to 0.0.0.0 8111 port [tcp/*] succeeded!
$ nc -zv ckerr-laptop 8111    # this name is listed in /etc/hosts, it illustrates that all of 127.0.0.0/24 will route to 127.0.0.1
Connection to ckerr-laptop (127.0.1.1) 8111 port [tcp/*] succeeded!

PS. If you're curious as to why connecting to why 0.0.0.0 might work (as someone practiced in the Sockets API I was surprised myself by this), the following has some good discussion and references to RFC5735 section 3. https://unix.stackexchange.com/questions/419880/connecting-to-ip-0-0-0-0-succeeds-how-why

[jeffbski-rga]

I was also running into the Error: connect ECONNREFUSED 127.0.0.1:8200 but none of the above seemed to make any difference.

When I fired up nc 127.0.0.1 8200 it would connect just fine, so I am guessing that maybe it is taking longer for dynamodb local to be actually ready.

I tried adding a 2s sleep after starting dynamodb-local before I started querying and it seems to be working.

So ultimately I took out the delay and just put in a retry that handles ECONNREFUSED on the first query after starting. This seems to be working fine and seems to take about 2s after launching to be ready.

I wonder if the old AWS SDK V2 had some built-in retries for ECONNREFUSED that is not in V3? Otherwise it doesn't make much sense to me that V2 would work fine without any special handling.

[cortexcompiler]

I know this is old, but wanted to point folks here: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/migrating/notable-changes/

Which says:

"If you are passing custom endpoint which uses http, then you need to provide httpAgent."

It would look something like (note that node-http-handler has moved to @smithy):

import * as https from 'https';
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { NodeHttpHandler } from '@smithy/node-http-handler';

const dynamodbClient = new DynamoDBClient({
  requestHandler: new NodeHttpHandler({
    httpAgent: new https.Agent({
      keepAlive: true,
    }),
    connectionTimeout: 30000,
    socketTimeout: 30000,
  }),
  endpoint: "http://host.docker.internal:4566",
});

[five00miles]

i use my local ipv4 like 192.168.0.2 (ifconfig on window)

  this.ddbClient = new DynamoDBClient({
    endpoint: "http://192.168.0.2:9000",
    logger: console
  });

[JustinLivi]

I'm currently encountering this and it appears to be an interaction with the SDK preferring environment variables over the code configurations. my configuration looks like

const client = new DynamoDBClient({
      endpoint: 'http://host.docker.internal:8000',
      accountIdEndpointMode: 'disabled',
      region: 'localhost',
      logger,
})

But when attempting to connect I get the error

getaddrinfo ENOTFOUND 123456789012.ddb.localhost.amazonaws.com

Logging my process.env I see

AWS_DEFAULT_REGION: "localhost"
AWS_REGION: "localhost"
AWS_ACCOUNT_ID: "123456789012"

I'm trying to connect locally from a lambda running in SAM against DynamoDB running in a local docker container. Is there any way to override the dynamically constructed endpoint going off the environment variables? As a client I would expect my code to take precedence, not my environment variables, at least if I'm hardcoding a specific endpoint. My environment variables might be being used for something entirely unrelated as that's a global configuration. I can see how this logic would be useful in the AWS environment but right now trying to run locally I'm fairly stuck.

[JustinLivi]

Just as a test I tried deleting those environment variables in my startup code.

delete process.env.AWS_DEFAULT_REGION;
delete process.env.AWS_REGION;
delete process.env.AWS_ACCOUNT_ID;

I then got the error

Region is missing

Adding the AWS_REGION environment variable back in I got

getaddrinfo ENOTFOUND dynamodb.localhost.amazonaws.com

I also tried with the endpoint configuration format used in the above thread with both combinations of environment variables but this encountered the same issues as previously described.

const client = new DynamoDBClient({
    endpoint: {
        hostname: 'host.docker.internal',
        port: 8000,
        path: '',
        protocol: 'http:',
    },
});

It appears the endpoint configuration does not do anything at all? Not sure how it's possible to get the code working as described earlier in this thread.

I'm on version 3.682.0 of @aws-sdk/client-dynamodb

[JustinLivi]

It appears that there is in fact a way to get past this problem if you lean into the environment variable based way of configuration. Using the service-specific endpoint environment variable AWS_ENDPOINT_URL_DYNAMODB worked for me.

More information:

• https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html
• https://docs.aws.amazon.com/sdkref/latest/guide/ss-endpoints-table.html

The other documented environment variables like AWS_ENABLE_ENDPOINT_DISCOVERY do not seem to function as documented, as setting this to false appears to have no effect.

[VasilyVP]

It needs to be used with { endpoint: 'http://host.docker.internal:8000' } instead of localhost I suppose it happens because a client makes a call inside a container and localhost targets to the container itself. A local lambda is called inside a container as well.

[aBurmeseDev]

Hi everyone, I've looked through all the previous comments, and it seems there isn't a single common issue here. Each case appears to be different, so there's nothing specific for our SDK team to fix. If you're still running into problems, please open a new issue and describe exactly what you're trying to do and what's not working. This will help us better track and address your specific situation.

[shellscape]

@aBurmeseDev that's absolutely hilariously classic AWS SDK team. here's a thought - start with the OP that offered a reproduction, and then ask people whose variants aren't solved by that resolution to open issues.

closing the issue wholesale because the sdk is so broken in this aspect, as to produce so many variants on an issue, without addressing the original issue provided isn't just lazy, it's being a bad steward of open source.