Program stops after some time with ETIMEDOUT error.

[vdualb]

i created simple bot with posts and privateMessage handlers. After running for some time it closes with the following error:

node main.js    
Starting bot
Initializing DB
(node:89951) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Logging in
Logged in

/home/dell/dbx-home/image_compare/node_modules/node-fetch/lib/index.js:1501
			reject(new FetchError(`request to ${request.url} failed, reason: ${err.message}`, 'system', err));
			       ^
FetchError: request to https://lemmy.world/api/v3/post/list?type_=Subscribed&auth=*censored*&sort=New failed, reason: 
    at ClientRequest.<anonymous> (/home/dell/dbx-home/image_compare/node_modules/node-fetch/lib/index.js:1501:11)
    at ClientRequest.emit (node:events:519:28)
    at TLSSocket.socketErrorListener (node:_http_client:495:9)
    at TLSSocket.emit (node:events:519:28)
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
    at runNextTicks (node:internal/process/task_queues:64:3)
    at listOnTimeout (node:internal/timers:540:9)
    at process.processTimers (node:internal/timers:514:7) {
  type: 'system',
  errno: 'ETIMEDOUT',
  code: 'ETIMEDOUT'
}

Node.js v21.5.0

It works for 20-30 mins before closing. One time it closed right after starting.

[SleeplessOne1917]

Judging by the stacktrace, the issue is with node-fetch. I found this issue on that project's issue tracker that has a similar stack trace. Is your bot making a lot of requests in a short time frame?

[vdualb]

@SleeplessOne1917 i set secondsBetweenPolls to 120. it's bigger than default value. you can take a look at the code: https://gist.github.com/RocketRide9/1589a7733f0be41e5d8ae7dda51dd0ac

[vdualb]

i removed async keywords from handlers and bot is running for 4 or so hours. i used it for await keyword. interesting

EDIT: it just crashed again

[SleeplessOne1917]

After looking into ETIMEDOUT more it seems like it's either the client or the server closing the connection before a response is returned. I bet lemmy.world being one of the most active lemmy instances contributes to this.

When you get the timedout errors, is it always when the bot is fetching subscribed posts like in the error you shared? Or has this been happening for other requests as well?

[vdualb]

@SleeplessOne1917 also happened with private message:

(.venv) ⬢ image_compare node main.js
Starting bot
Initializing DB
(node:266273) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Logging in
Logged in

/home/dell/dbx-home/image_compare/node_modules/node-fetch/lib/index.js:1501
			reject(new FetchError(`request to ${request.url} failed, reason: ${err.message}`, 'system', err));
			       ^
FetchError: request to https://lemmy.world/api/v3/private_message/list?auth=censored&limit=50&unread_only=true failed, reason: 
    at ClientRequest.<anonymous> (/home/dell/dbx-home/image_compare/node_modules/node-fetch/lib/index.js:1501:11)
    at ClientRequest.emit (node:events:519:28)
    at TLSSocket.socketErrorListener (node:_http_client:495:9)
    at TLSSocket.emit (node:events:519:28)
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  type: 'system',
  errno: 'ETIMEDOUT',
  code: 'ETIMEDOUT'
}

Node.js v21.5.0

i use lemmy-bot 0.4.6 btw

[SleeplessOne1917]

Do you fare any better if you try it with version 0.5.1?

[vdualb]

world is running lemmy 18 so i cant use 0.5.x (according to README). is lemmy.ml good enough to test bot there?

[SleeplessOne1917]

lemmy.ml should be good to test. In addition to the newer API, I'm curious if lemmy.ml's lower traffic (compared to world) will also affect the timeouts.

[vdualb]

i finally found an instance where my registration wasn't denied. i changed lemmy-bot version to 0.5.1 and changed credentials. allow list doesn't seem to work. I added this line to post handler body:

console.log("recieved new post:", ap_id);

In ~5 mins after starting the bot I see this line recieved new post: https://slrpnk.net/post/6092668. It isn't a post from community I set in allow list.

[vdualb]

btw can bot try again later if server didn't respond? I think websites getting down isn't something uncommon

[SleeplessOne1917]

btw can bot try again later if server didn't respond? I think websites getting down isn't something uncommon

Now that you mention it, I don't think I handle that case very well. I will take a look and make a change.

[vdualb]

I am using this script as a temporary solution:

#/bin/bash

echo "Building..."
tsc
echo "Starting bot..."
node main.js
while true
do
  echo "Restarting..."
  sleep 120
  node main.js
done

So far working great