Error: ENOENT: no such file or directory, rename

[joshmanders]

I keep periodically getting this error. Using [email protected].

Error: ENOENT: no such file or directory, rename '[...]/node_modules/geolite2-redist/dbs.geodownload/GeoLite2-City.mmdb' -> '[...]/node_modules/geolite2-redist/dbs/GeoLite2-City.mmdb'
    at Object.renameSync (node:fs:1032:11)
    at [...]/node_modules/geolite2-redist/dist/download-helpers.js:87:16

[GitSquared]

Thanks for the report Josh

Looks like the library is failing to download a database update, are you running behind a firewall by any chance? If so, you might have to whitelist some IPs so it can reach the GitHub mirror – see the last bit of this section of the README

Let me know if that fix things up!

---

Affected code is here for ref: https://github.com/GitSquared/node-geolite2-redist/blob/acf9b7e98f949f9930ae44da0af52b4f4542d4c8/src/download-helpers.ts#L141

[joshmanders]

Nope no firewall afaik, it happens periodically both locally in development and in production.

[natanavra]

+1 started getting this error today we've been working with the package for several years

[GitSquared]

@natanavra could you give me a bit more info about the environment you're running the package in?

Possible root causes include bad network access, missing write permissions on disk. I checked and the redist tarballs are properly formatted.

Today was the first redist update in 3 months, so anything that might have changed in your infra since 3 months ago might be a good lead to follow

[natanavra]

@GitSquared No changes in infra whatsoever Amazon Linux / Ubuntu 20 + NodeJS 16-18 (several environments)

Been getting this error on several projects starting this morning

Specifically: ENOENT: no such file or directory, rename '.../node_modules/geolite2-redist/dbs.geodownload/GeoLite2-City.mmdb' -> '.../node_modules/geolite2-redist/dbs/GeoLite2-City.mmdb'

[natanavra]

Something very bad is going on with the db download Getting more errors: Checksum mismatch for GeoLite2-City ENOENT: Cannot cd into '/home/ubuntu/server/node_modules/geolite2-redist/dbs.geodownload

@GitSquared Can you rollback the deployed version?

This is breaking the whole process right, and there isn't much that can be done, it's throwing unhandled promise rejections The whole auto-update operation should be wrapped and send out warnings instead

[GitSquared]

@natanavra I just released https://github.com/GitSquared/node-geolite2-redist/releases/tag/v3.1.0 which should improve error and filesystem handling.

Let me know if that improves stuff on your end. I couldn't manage to reproduce the issue you're having while doing some tests locally.

[natanavra]

@GitSquared Thanks! I worked around it in the meantime by manually updating the db file in the dist folder

[natanavra]

May I suggest adding an event listener instead of throwing an error? Or even adding a callback parameter that could be passed and called instead of throwing an error. If there's a callback - great call it If not - throw an error

[GitSquared]

Yes that's a good idea! Will implem whenever I have time.

[natanavra]

This keeps happening: Failed to replace GeoLite2-City with updated version: Error: ENOENT: no such file or directory, rename 'node_modules/geolite2-redist/dbs-tmp/GeoLite2-City.mmdb' -> 'node_modules/geolite2-redist/dbs/GeoLite2-City.mmdb'

[natanavra]

Could be related to Github rate limit on downloading the db file? 🤔 I just tried to wget the GeoLite2-City db from the mirror URL (github raw file) and the download dropped in the middle (it did succeed after a few minutes), since the download just happens on public access URL on github, it's possible they're rate limiting access to the file The download fails and this breaks the AutoUpdater process

Edit 1: Github docs: The primary rate limit for unauthenticated requests is 60 requests per hour.

Theoretically with enough projects running on shared instances you could hit the rate limit

Edit 2: got documentation mentions there will be no errors on read unless using new stream.PassThrough(), that also supports the fact the file was not downloaded, and untar throwing a "no such file" error

[GitSquared]

Thanks for digging in @natanavra, this is super useful!

Could you confirm those errors are not crashing your app anymore?

What's really bothering me is this:

we've been working with the package for several years

I don't understand what could have possibly changed for you to start running into those issues. There were no redist updates between end of April and last week (~3 months) because of a CI failure due to external changes, that I didn't detect very timely – fixed my notifications setup since. But before that, the package was steadily updating its redist every ~2 days since at least 2022.

Rate limits are a good candidate indeed, if you have 60+ Node processes running behind a single IP (load balancer?).

[natanavra]

@GitSquared The errors did not stop, only when I update the file manually. I've since worked around this by monkey-patching the package and stopping AutoUpdater.

Nothing changed, that's the thing, it's really weird since we've being using this package since probably its inception with no issues whatsoever. That's why I'm theorizing this is a rate limit issue + got default error-less behavior on Read stream.

We have sometimes over 60 nodes running in a cluster, although they have different start times, it's still possible this is the issue. Moreover, the underlying hardware and network in AWS EC2 is shared and there's probably only several gateways to the internet from a single AWS region. In which case multiple unrelated applications using this package might go through the same external gateway and have the same public IP and all requests/downloads count towards the same ratelimit on Github.

EDIT: A temporary solution for now would be to check the file existance here before proceesing with the tar op, e.g. with something like

fs.existsSync(hotDownloadDir);